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1.0  Introduction 


This  report  summarizes  the  work  accomplished  under  Gram  #  AFOSR-88-0024  during  the 
time  period  1  October,  1987  through  February  28, 1991.  The  summary  presented  here  is  a 
brief  one.  The  details  of  the  results  can  be  found  in  the  many  publications  that  resulted 
from  the  work  supported  here. 

The  work  carried  out  fell  in  two  specific  areas:  1)  Theory  of  neural  networks,  and  2)  Fun¬ 
damental  properties  of  optical  interconnections.  Two  students  received  their  Ph.D.’s  under 
this  support,  one  in  each  area.  We  summarize  the  results  in  each  of  these  areas  in  what  fol¬ 
lows. 


2.0  Theory  of  Neural  Networks 

The  work  in  this  area  was  purely  theoretical,  and  focused  on  fundamental  properties  of  the 

Hopfield  network.  The  Ph.D.  student  responsible  for  this  work  was  Jehosua  Bruck,  who  is 

now  working  at  the  IBM  Almaden  Research  Laboratory.  Particular  results  fell  in  several 

different  sub-areas,  as  now  described. 

2.1  Hopfield  Network  as  an  Associative  Memory 

•  We  demonstrated  that  the  capacity  of  the  Hopfield  network  as  an  associative  memory  is 
extremely  limited.  For  example,  it  is  impossible  to  store  and  successfully  retrieve  two 
vectors  at  Hamming  distance  1 .  For  vectors  having  a  Hamming  distance  greater  than  1 , 
the  maximum  storage  capacity  is  less  than  4  vectors. 

•  We  demonstrated  that  the  network  should  be  fully  connected.  There  are  some  very  sim¬ 
ple  tasks  that  the  network  can  not  perform  if  it  is  not  fully  connected. 

•  We  demonstrated  that  mappings  that  contain  a  sphere  (such  as  the  decoder  for  a  Ham¬ 
ming  code)  can  not  be  realized  by  such  a  network. 

•  We  showed  that  the  outer  product  method  for  programming  memory  can  result  in  many 
(exponential  in  the  number  of  words  we  wish  to  store)  spurious  states. 

2.2  Hopfield  Network  for  Performing  “Hard”  Computations 

•  We  showed  that  even  if  we  let  the  network  run  for  an  exponential  time,  we  still  need  an 
exponential  number  of  neurons  to  solve  a  hard  (NP  complete)  problem.  This  conclusion 
is  also  true  if  we  are  satisfied  to  obtain  an  approximate  solution,  as  opposed  to  an  exact 
solution. 

2.3  What  Problems  is  the  Hopfield  Network  Well-Suited  For? 

•  Optimization  problems  must  be  mapped  to  the  network  by  formulating  them  as  a  maxi¬ 
mization  of  a  set  of  quadratic  forms  that  are  coupled  by  heuristically  determined  coeffi¬ 
cients. 
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•  Finding  the  global  maximum  of  the  energy  function  associated  with  the  network  run¬ 
ning  in  a  serial  mode  is  equivalent  to  finding  the  minimum  cut  in  the  undirected  graph 
associated  with  the  network. 

•  The  maximum  likelihood  decoding  problem  for  linear  block  codes  is  equivalent  to  find¬ 
ing  the  global  maximum  os  the  energy  function  of  a  network  that  can  be  constructed 
knowing  the  generator  matrix  of  the  code. 

•  Given  a  linear  block  code,  a  network  can  be  constructed  using  the  parity  check  matrix, 
such  that  every  stable  state  of  the  network  corresponds  to  a  codeword  and  every  code¬ 
word  corresponds  to  a  stable  state,  thus  solving  the  “programming”  problem  for  linear 
codes. 

2.4  Can  the  Power  of  Such  Networks  be  Increased  by  Increasing  the 
Complexity  of  an  Individual  Neuron? 

•  Normally  neurons  are  considered  to  be  simple  linear  threshold  devices.  If  they  are 
increased  in  complexity  to  be  polynomial  threshold  devices,  does  the  power  of  the  net¬ 
work  increase?  The  answer  is  that  not  much  is  gained.  A  two  layer  network  of  linear 
threshold  elements  is  strictly  more  powerful  than  a  single  polynomial  threshold  ele¬ 
ment 

2.5  Publications  on  Neural  Networks  Resulting  from  Grant  Support 

Published 

•  J.  Bruck  and  J.  Sanz,  “A  study  on  Neural  Networks”,  International  Journal  of  Intelli¬ 
gent  Systems,  Vol.  3,  pp.  59-75  (1988). 

•  J.  Bruck  and  J.W.  Goodman,  “A  generalized  convergence  theorem  for  neural  net¬ 
works”,  IEEE  Trans,  on  Info.  Theory,  Vol.  34,  pp.  1089-1092  (1988) 

•  J.  Bruck  and  J.W.  Goodman,  “On  the  power  of  neural  networks  for  solving  hard  prob¬ 
lems”,  Journal  of  Complexity,  Vol.  6,  pp.  129-135  (1990). 

•  J.  Bruck  and  M.  Blaum,  “Neural  networks,  error  correcting  codes  and  polynomials  over 
the  binary  n-cube”,  IEEE  Trans,  on  Info.  Theory,  Vol.  35,  pp.  976-987  (1989). 

•  J.  Bruck,  “Harmonic  analysis  of  polynomial  threshold  functions”,  SIAM  Journal  on 
Discrete  Mathematics,  \bl.  3,  pp.  168-177  (1990). 

•  J.  Bruck  and  V.P.  Roychowdhury,  “On  the  number  of  spurious  memories  in  the  Hop- 
field  model”,  IEEE  Trans,  on  Info.  Theory,  Vol.  36,  pp.  393-397  (1990). 

•  J.  Bruck,  “On  the  convergence  properties  of  the  Hopfield  model”,  Proc.  IEEE,  Special 
Issue  on  Neural  Networks,  Vol.  78,  pp.  1579-1585  (1990). 

•  J.  Bruck,  Computing  with  networks  of  threshold  elements,  Ph.D.  Thesis,  Department  of 
Electrical  Engineering,  Stanford  University  (1989). 
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3.0  Fundamental  Properties  of  Optical  Interconnections 

The  work  in  this  area  was  again  purely  theoretical,  and  focused  on  fundamental  properties 
of  various  interconnect  technologies,  including  optical  interconnects,  normally  conducting 
electrical  interconnects,  and  superconducting  electrical  interconnects.  Results  in  the  area 
of  thermal  limitations  of  electrical  interconnects  were  also  obtained.  The  work  in  this  area 
was  carried  out  by  Mr.  Haldun  Ozaktas,  who  will  receive  his  Ph.D.  degree  this  June.  Mr. 
Ozaktas  has  not  yet  accepted  a  job  following  his  Ph.D.  conferral.  A  summary  of  the  major 
contributions  follows. 

3.1  Interconnect  Dominated  Model  of  Computation 

•  Developed  a  basic  theory  governing  wireability  limitations  to  system  size.  Showed  that 
wiring  area  ultimately  limits  the  minimum  size  of  two-dimensional  systems. 

•  Determined  heat  removal  limitations  to  system  size.  Showed  that  heat  removal  ulti¬ 
mately  limits  the  volume  of  three-dimensional  systems. 

3.2  Optical  Systems 

•  Determined  a  lower  bound  for  the  communication  volume  required  for  an  optically 
interconnected  array  of  points. 

•  Examined  several  different  interconnect  architectures  have  been  proposed  for  imple¬ 
mentation  in  optics.  The  multifacet  holographic  interconnect  architecture  (based  on 
ffee-space  propagation  to  a  faceted  hologram)  and  the  folded  multi-facet  architecture 
(based  on  substrate  mode  holograms)  were  shown  to  have  less  than  optimum  two- 
dimensional  growth  properties,  and  an  improvement  with  close  to  optimum  growth  in 
two  dimensions  was  found. 

•  A  three-dimensional  multifacet  architecture  with  optimum  volume  growth  properties 
was  also  found. 

•  The  optimal  electromagnetic  carrier  frequency  that  balances  metrical  information  den¬ 
sity  and  heat  removal  requirements  was  derived  and  found  to  be  in  the  general  vicinity 
of  the  visible-to-near-IR  region  of  the  spectrum. 

3.3  Physical  Limitations  to  Communication  in  Computation 

•  Determined  the  fundamental  properties  of  optical,  normally  conducting  electrical,  and 
superconducting  electrical  interconnections.  In  particular,  the  interaction  of  data  rate, 
latency,  and  number  of  processors  was  explored,  based  on  both  wireability  and  heat 
removal. 

•  Examined  hybrid  interconnect  architectures,  in  which  normally  conducting  electrical 
interconnects  are  used  for  short  distance  communication,  and  either  optics  or  supercon¬ 
ducting  electrical  interconnects  is  used  for  long  distance  communication.  Determined 
the  optimal  partitioning  of  interconnect  technologies  in  a  given  computational  struc¬ 
ture. 
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•  Examined  the  concept  of  the  all-optical  digital  computer,  in  which  both  the  intercon¬ 
nects  and  the  logic  elements  are  optical.  Determined  the  number  of  processors  needed 
before  the  all-optical  computer  is  competitive  with  a  hybrid-interconnected  electronic 
computer.  Determined  the  likely  characteristics  of  an  all-optical  computer,  in  terms  of 
numbers  of  processors  and  volume  occupied. 

•  Examined  some  indirect  implementations  of  optical  interconnects,  including  the  multi¬ 
plexed  grid  architecture  and  the  multiplexed  fat  tree  architecture. 

Publications  on  Optical  Interconnects  Resulting  from  Grant  Support 

Published  and  Accepted: 

•  H.M.  Ozaktas  and  J.W. Goodman,  “Lower  bound  for  the  communication  volume 
required  for  an  optically  interconnected  array  of  points”,  J.  Opt.  Soc.  Am.  -  A ,  Vol.  7,  pp. 
2100-2106(1990). 

•  H.M.  Ozaktas,  Y.  Amitai,  and  J.W.  Goodman,  “Comparison  of  system  size  for  some 
optical  interconnection  architectures  and  the  folded  multi-facet  architecture”.  Opt. 
Commun.,  1991.  (Accepted  for  publication). 

•  H.M.  Ozaktas,  Y.  Amitai,  and  J.W.  Goodman,  “A  three  dimensional  optical  intercon¬ 
nection  architecture  with  minimal  growth  rate  of  system  size”.  Opt.  Commun.,  1991. 
(Accepted  for  publication) . 

•  H.M.  Ozaktas  and  J.W.  Goodman,  “The  limitations  of  interconnections  in  providing 
communication  between  an  array  of  points”,  in  Frontiers  of  Computing  Systems 
Research  (Stuart  K.  Tewksbury,  Editor),  Plenum  Press,  New  York,  1991.  (In  Press) 

Submitted: 

•  H.M.  Ozaktas  and  J.W.  Goodman,  “The  optimal  electromagnetic  carrier  frequency  bal¬ 
ancing  structural  and  metrical  information  densities  with  respect  to  heat  removal 
requirements”,  J.  Opt.  Soc.  Am.  -  A.  (Submitted  1990.) 

•  H.M.  Ozaktas,  H.  Oksuzoglu,  R.F.W.  Pease,  and  J.W.  Goodman,  “The  effect  on  scaling 
of  heat  removal  requirements  in  3  dimensional  systems”,  IEEE  Electron  Dev.  Lett. 
(Submitted  1990). 

In  preparation: 

•  H.M.  Ozaktas  and  J.W.  Goodman,  “Organization  of  information  flow  in  computation 
for  efficient  utilization  of  high  information  flux  communication  media”,  (To  be  submit¬ 
ted  to  Applied  Optics  or  Optics  Communications.) 

•  H.M.  Ozaktas  and  J.W.  Goodman,  “On  the  usefulness  of  optical  digital  computing”, 
(To  be  submitted  to  Applied  Optics.) 

•  H.  M.  Ozaktas,  A  Physical  Approach  to  Communications  Limits  in  Computing,  Ph.D. 
Thesis,  Department  of  Electrical  Engineering,  Stanford  University,  June  1991. 
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Conference  papers: 

•  H.M.  Ozaktas  and  J.W.  Goodman,  “Optimal  partitioning  of  very  large  scale  optoelec¬ 
tronic  computing  systems”,  In  1991  Annual  Meeting  Technical  Digest,  Optical  Society 
of  America,  1990. 

•  H.M.  Ozaktas  and  J.W.  Goodman,  “Multiplexed  hybrid  interconnection  architectures”, 
In  Technical  Digest  of  the  1991  OS  A  Topical  Meeting  on  Optical  Computing,  1991. 
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